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The  light  which  reaches  the  eye,  or  any  other  sensor,  is  the  product  of  the  reflectance  and  the 
illuminant.  Therefore,  in  order  to  determine  the  surface  reflectance  of  an  object  independent  of  the 
illuminant,  a  system  must  use  the  spatiochromatic  context  of  the  image.  We  have  developed  a  neural 
network  based  on  the  anatomy  and  physiology  of  the  visual  projection  from  retina  to  V4.  The  network 
combines  color-opponent  and  contrast  information  to  achieve  a  good  degree  of  color  constancy.  This 
network  has  been  tested  on  simulated  images  corresponding  to  the  stimuli  used  in  well  established 
psychophysical  experiments.  Responses  qualitatively  match  human  responses  to  a  variety  of  center- 
surround  and  Mondrian  test  stimuli. 


INTRODUCTION: 

Color  constancy  is  the  ability  to  maintain  an  approximately  constant  color  perception  despite  changes  in 
the  incident  illumination  of  the  object.  Color  contrast,  also  referred  to  as  chromatic  induction  or  simultaneous 
contrast,  is  the  change  in  the  (perceived)  color  of  a  surface  due  to  the  spectral  composition  of  neighboring  sur¬ 
faces.  Color  perception  in  natural  scenes  depends  upon  both  of  these  phenomena.  Together,  these  two  effects 
demonstrate  that  color  perception  does  not  directly  depend  upon  the  wavelength  of  the  light  reflected  from  a 
surface. 

Many  theories  for  color  constancy  and  color  contrast  have  been  proposed  previously  (e.g.  D'Zmura  and 
Lennie,  1986;  Maloney  and  Wandell,  1986).  However,  they  usually  have  required  unrealistic  assumptions  about 
images,  had  only  loose  connection  to  anatomy  and  physiology,  or  required  that  parameters  be  changed  for  each 
image.  The  dependence  of  color  constancy  and  color  contrast  on  the  spatiochromatic  properties  of  a  large  portion 
of  the  image  (e.g.  Walraven,  1976;  Blackwell  and  Buchsbaum,  1988a,b;  review  by  Jameson  and  Hurvich,  1989) 
suggests  that  a  distributed  method  of  processing,  such  as  that  provided  by  neural  networks,  could  be  very  useful  in 
solving  this  problem. 

Several  neural  network  models  have  been  proposed.  For  example,  Grossberg  (1987)  proposed  a  model  in 
which  color  information  is  obtained  from  the  contrast  at  boundaries  and  then  the  color  is  filled  into  each  segment 
of  the  image.  Dufort  and  Lumsden  (1991)  used  double-opponent  cells  as  a  mechanism  for  color  constancy.  Usui, 
Nakauchi,  and  Miyake  (1990)  developed  a  neural  network  which  transformed  color  signals  from  cone  signals  to 
the  very  narrowly  tuned  color  responses  which  Zeki  (1980, 1983)  reported  finding  in  V4.  They  trained  this  net¬ 
work  using  back-propagation  and  found  that  the  hidden  units  resembled  the  color  opponent  cells  in  the  retina  and 
LGN. 

One  of  the  best  known  algorithms  for  color  constancy.  Land's  retinex  algorithm  (Land  1964, 1986),  cal¬ 
culates  the  relative  "lightness"  of  each  area  of  a  scene  within  three  separate  channels,  each  sensitive  to  a  different 
region  of  the  visual  spectrum.  Moore,  Allman,  and  Goodman  (1991)  implemented  the  retinex  algorithm  within  a 
network  structure.  One  limitation  of  the  retinex  is  that  the  interiois  of  large  uniform  regions  become  gray.  Moore 
et  al.  (1991)  incorporated  a  modification  in  order  to  eliminate  this  washout  problem  by  multiplying  the  surround 
factor  by  the  "edginess"  of  the  area.  Hurlbert  and  Poggio  (1988)  demonstrated  that,  using  a  number  of  different 
"learning"  methods,  including  least  squares  gradient  descent  and  back  propagation  in  neural  networks,  a  linear 
operator  could  be  found  which  is  similar  to  that  proposed  in  the  retinex  algorithm. 

One  of  the  most  fundamental  problems  with  many  of  these  networks  is  that  because  they  record  only 
contrast,  they  require  assumptions  about  the  average  to  which  that  contrast  refers.  A  common  assumption  made  is 
the  "gray  world*  assumption,  which  states  that  the  average  cbromaticity  is  constant  for  all  inuges.  This  was 
developed  formally  by  Buchsbaum  (1980)  and  has  been  used  in  many  color  constancy  algorithms  since.  Other 
assumptions  include  that  the  brightest  spot  in  an  image  is  white,  or  that  the  illumination  varies  slowly  in  space. 
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while  reflectances  have  sharp  chromatic  borders.  These  assumptions  hold  for  many  scenes,  but  when  they  do  not 
hold  these  algorithms  can  make  predictions  far  from  the  psychophysical  results.  In  the  following  section,  we 
describe  how  our  model  eliminates  both  the  washout  problem  and  the  need  for  assumptions  about  the  scene  by 
measuring  the  baseline  as  well  as  the  contrast. 

THE  MODEL: 

We  have  develop>ed  and  simulated  a  neural  network  model  that  provides  a  biologically  plausible  neural 
architecture  capable  of  producing  color  constancy  and  color  contrast.  The  model  is  based  on  primate  anatomical 
and  physiological  data  from  the  retina  to  V4  and  incorporates  both  classical  center-surround  inhibition  as  is  found 
in  the  retina  and  LGN,  and  the  extremely  wide-field  "silent  surround"  inhibition  reported  in  V4  (Schein  and 
Desimone,  1990).  As  is  shown  in  figure  1,  the  simulation  includes  a  preprocessor  which  converts  an  image  from 
either  CIE  coordinates  or  Munsell  notation  to  the  cone  activities:  R(x,y),  G(x,y),  B(x,y).  These  three  matrices  of 
activation  levels  comprise  the  input  to  the  network. 


Overview  of  the  Model 
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Figure  I:  Overview  of  the  network  model  showing  connection  patterns  at  each  stage.  See  text  for  details. 

The  input  is  then  fed  to  a  center-surround  color  opponent  stage.  Each  of  the  three  color  pathways  con¬ 
sists  of  an  on-center  and  an  off-center  pathway  whose  receptive  fields  are  defined  by  a  difference  of  Gaussians. 
The  center  receives  input  from  a  single  cone.  The  surround  receives  input  from  multiple  cone  types.  The  center 
and  surround  arc  sensitive  to  different  wavelengths  and  so  arc  called  color  opponent.  The  center-surround  stage 
provides  the  input  for  the  large  "silent  surround"  stage  which  is  based  on  the  physiological  results  of  Schein  and 
Desimone  (1990)  and  Zcki  (1980,  1983)  from  area  V4  in  the  macaque  cortex.  &ch  of  these  units  has  a  large  (up 
to  16°)  surround  outside  of  its  classical  receptive  field  which  is  tuned  to  approximately  the  same  wavelength  as  its 
center,  unlike  the  color  opponent  cells  in  the  previous  layer.  The  connection  strength  of  the  inputs  to  these  sur¬ 
rounds  decreases  with  increasing  distance  from  the  center. 
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Wc  refer  to  these  units  with  large  surrounds  as  contrast  cells.  We  define  "positive  contrast”  as  the  center 
being  more  active  than  the  surround  and  "negative  contrast"  as  the  surround  being  more  active  than  the  center. 
Positive  contrast  cells  receive  input  from  on-center  cells  and  negative  contrast  cells  receive  input  from  off-center 
cells.  However,  if  the  network  measured  only  contrast,  it  would  require  the  gray  world  assumption,  as  many  of 
the  previous  models  did.  Therefore,  this  network  model  calculates  the  "baseline"  level  as  well  as  the  contrast. 

The  "baseline"  is  a  local  average  of  the  color  opponent  response.  This  is  consistent  with  the  physiological  results 
of  Schein  and  Desimone  (1990)  who  found  that,  while  most  of  the  cells  in  V4  did  have  silent  surrounds,  there  was 
also  a  signiHcant  quantity  that  did  not  have  these  surrounds.  Therefore  the  baseline  is  calculated  by  units  based  on 
the  classical  receptive  fields  of  V4  cells. 

Figure  2  shows  how  the  contrast  and  baseline  information  are  combined  in  this  model  to  create  a  single 
representation  of  "color".  In  the  case  where  both  positive  and  negative  contrast  arc  zero  (i.c.  the  case  of  a  large 
uniform  field)  the  output  should  follow  the  baseline  cells.  This  avoids  the  washout  problem.  However,  if  the  sur¬ 
round  response  is  less  than  the  center  (positive  contrast)  the  output  is  enhanced  above  the  base  activity,  whereas  if 
the  surround  response  is  more  than  the  center  (negative  contrast)  then  the  output  is  reduced  below  the  base  activ¬ 
ity.  This  follows  the  psychophysical  result  that  a  gray  square  looks  darker  against  a  bright  background  than 
against  a  dark  background.  This  calculation  is  done  separately  in  each  of  the  color  channels.  As  is  shown  in 
figure  1,  the  contrast  and  baseline  calculations  are  done  only  for  the  center  of  the  input  image  for  simplicity  and 
minimized  run-time. 


baseline 


output 


Figure  2:  The  output  for  each  color  channel  is  the  baseline  response  plus  the  difference  between  the  positive  and 
negative  contrasts.  See  figure  1. 

THE  SIMULATION: 

The  network  was  simulated  using  NEXUS,  a  biologically  based  neural  network  simulation  program  writ¬ 
ten  at  the  University  of  Pennsylvania  by  Paul  Sajda  and  Leif  Finkel  (1992).  NEXUS  is  designed  for  creating  mul¬ 
tiple,  large-scale  neural  maps  and  provides  considerable  flexibility  in  assigning  physiological  properties  and 
anatomical  architectures.  The  program  is  written  in  C  and  features  a  graphical  user  interface  (X-window  based) 
which  makes  setting  up  networks  straightforward.  Any  network  variable  can  be  monitored  in  display  windows, 
and  a  number  of  diagnostic  tools  are  available. 

The  simulation  was  tested  for  color  contrast  response  using  center-surround  images  with  varying  separa¬ 
tion  between  the  center  and  the  surround.  Blackwell  and  Buebsbaum  (1988a)  have  shown  that  the  size  of  the 
color  shift  decreases  as  the  distance  between  center  and  surround  increases.  As  is  shown  in  figure  3,  the  network 
simulation  also  shows  this  behavior.  This  is  a  consequence  of  the  weighting  function  of  the  silent  surrounds  in  the 
V4  stage  of  the  model. 

The  simulation  was  also  tested  for  its  color  constancy  ability  using  the  classical  McCann  Mondrian  dem¬ 
onstration  (McCann,  McKee,  and  Taylor,  1976)  in  which  the  illuminant  on  one  Mondrian  image  is  changed  until 
the  physical  R,G,B  of  a  patch  in  that  Mondrian  is  identical  to  a  patch  with  different  reflectance  in  a  Mondrian  un¬ 
der  a  white  illuminant.  The  particular  case  that  we  used  was  to  change  the  illuminant  on  one  Mondrian  so  that  a 
colored  patch  had  the  same  physical,  input  R,G,B  as  a  gray  patch  in  the  Mondrian  under  the  white  illuminant.  A 
human  presented  with  this  situation  is  able  to  distinguish  the  colored  patch  from  the  gray  patch.  We  simulated  this 
experiment  with  the  network  using  several  different  colored  patches.  As  is  shown  is  figure  4,  the  network  is  able 
to  distinguish  the  colored  patch  from  the  gray  patch.  In  addition,  the  outputs  are  similar  (but  not  identical)  for  the 
same  patch  viewed  under  the  two  different  illuminants.  (Note  that  human  color  constancy  is  not  100%  effective 
either.)  Thus  the  network  demonstrates  the  major  properties  of  human  color  constancy. 


Test  for  Spatial  Properties  of  Color  Contrast  Response 


Figure  3:  Change  in  output  of  each  color  channel  for  a  bluegreen  patch  due  to  a  green  surround  and  a  blue 
surround  .  The  change  is  defined  as  the  difference  between  the  outputs  with  and  without  the  indicated  surround. 
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Figure  4:  (a,  previous  page)  The  experimental  setup.  The  left  Mondrian  has  a  gray  patch  in  the  center  and  a 
constant  white  illuminant.  The  right  Mondrian  has  a  test  patch  in  the  center  which  may  be  any  color.  Tlie 
intensities  of  the  three  colored  lights  on  the  right  Mondrian  are  changed  until  a  light  meter  registers  the  same  cone 
R,G,B  for  the  test  patch  as  for  the  gray  patch,  (b)  The  normalized  output  of  the  simulation's  three  color  channels 
in  V4  to  patches  of  different  reflectances  within  a  simulated  Mondrian.  The  top  set  of  data  for  each  colored  patch 
is  the  output  under  a  white  illuminant.  The  bottom  set  of  data  is  the  output  under  an  illuminant  which  gives  each 
patch  the  same  input  R,G,B  as  the  gray  patch  under  the  white  illuminant.  Therefore,  if  there  were  no  color 
constancy  effect,  the  outputs  in  the  bottom  graph  (altered  illuminant)  would  all  be  identical  to  the  output  for  gray. 

Conclusion: 

We  have  developed  a  neural  network  model  of  color  constancy  and  color  contrast  /hich  agrees  with  the 
psychophysical  data.  Because  it  uses  multiple  cell  types  to  retain  information  about  the  biscline  at  each  point  as 
well  as  the  contrast,  it  requires  no  assumptions  about  the  properties  of  images  and  dors  tot  suffer  from  the  wash¬ 
out  problem  that  the  retinex  algorithm  has.  Because  the  network  is  based  closely  o  .  components  of  the  visual 
system,  it  may  be  used  to  test  hypotheses  about  the  relative  significance  of  these  components  to  the  human  color 
constancy  abilities.  There  has  been  much  debate  over  the  relative  significance  of  cone  adaptation  versus  cortical 
processing  in  color  perception.  We  can  now  add  adaptation  to  the  network  and  compare  it's  contribution  to  that  of 
the  V4  stage.  Each  parameter  (e.g.  the  size  and  shape  of  each  receptive  field,  relative  connection  strengths  of  the 
base,  the  positive,  and  the  negative  contrast  cells  to  the  output)  may  be  manipulated  separately  and  the  output 
monitored. 

We  would  like  to  incorporate  two  additional  features  into  this  network.  The  network  does  not  currently 
use  any  learning  algorithms.  All  parameters  are  taken  from  physiological  data.  However,  we  believe  that  the 
large  receptive  fields  of  the  contrast  cells  could  be  developed  through  a  Hebbian  learning  process.  Also,  this 
network  would  be  more  useful  as  an  artificial  color  recognition  device  if  the  output  were  immediately  interpret- 
able  as  a  defined  color,  rather  than  as  an  activity  pattern  in  a  population  of  cells.  Therefore,  we  will  add  a 
parallel-distributed-processing  network  to  the  biolrgically  based  network  to  act  as  a  pattern  recognition  device. 
The  PDF  will  be  trained  on  images  of  single  co'or  patches  on  gray  backgrounds  under  a  standard,  achromatic 
illuminant.  Therefore,  the  changes  in  predicted  color  appearance  under  other  illuminants  and  using  other  back- 


grounds  will  be  due  to  the  biological  network,  not  the  PDP. 

Our  results  so  far  indicate  that  the  contrast  within  each  color  channel  (as  is  apparently  measured  in  V4  of 

the  visual  cortex)  is  the  most  important  parameter  affecting  color  constancy  and  color  contrast.  This  model 

addresses  how  the  contrast  is  used  and  what  other  information  is  needed. 
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